Definition

This chapter describes the processes involved in identifying and developing a topic for research investigation. It was suggested that researchers consider several sources for potential ideas, including a critical analysis of everyday situations. The steps in developing a topic for investigation naturally become easier with experience; beginning researchers need to pay particular attention to material already available. They should not attempt to tackle broad research questions, but should try to isolate a smaller, more practical subtopic for study. They should develop an appropriate method of analysis and then proceed, through data analysis and interpretation, to a clear and concise presentation of results.

The chapter stresses that the results of a single survey or other research approach only provide indications of what may or may not exist. Before researchers can claim support for a research question or hypothesis, the study must be replicated a number of times to eliminate dependence on extraneous factors.

While conducting research studies, investigators must be constantly aware of potential sources of error that may create spurious results. Phenomena that affect an experiment in this way are sources of breakdowns in internal validity. If and only, if differing and rival hypotheses are ruled out can researchers validly say that the treatment was influential in creating differences between the experimental and control groups. A good explanation of research results rules out intervening variables; every plausible rival explanation should be considered. However, even when this is accomplished, the results of one study can be considered only as indications of what may or may not exist. Support for a theory or hypothesis can be made only after the completion of several studies that produce similar results.

In addition, for a study to have substantive worth to the understanding of mass media, the results must be generalizable to subjects and groups other than those involved in the experiment. External validity can be best achieved through randomization of subject selection.

The scientific evaluation of any problem must follow a sequence of steps to increase the chances of producing relevant data. Researchers who do not follow a prescribed set of steps do not subscribe to the scientific method of inquiry and simply increase the amount of error present in the study. This chapter describes the process of scientific research, from identifying and developing a topic for investigation to replication of results. The first section briefly introduces the steps in the development of a research topic.

Objective, rigorous observation and analysis are characteristic of the scientific method. To meet this goal, researchers must follow the prescribed steps shown in Figure 2.1. This research model is appropriate to all areas of scientific research.

Figure 2.1: Steps in the development of a research project

2.1 Selecting a Research Topic 

Selecting a research topic is not a concern for all researchers; in fact, only a few investigators in communications fields are fortunate enough to be able to choose and concentrate on a research area interesting to them. Many come to be identified with studies of specific types, such as focus group methodology, magazine advertising, or communications and the law. These researchers investigate small pieces of a puzzle in communications to obtain a broad picture of their research area.

In the private sector, researchers generally do not have the flexibility of selecting topics or questions to investigate. Instead, they conduct studies to answer questions raised by management or they address the problems/questions for which they are hired, as is the case with full-service research companies.

Although some private sector researchers are limited in the amount of input they can contribute to topic selection, they usually are given total control over how the question should be answered; that is, what research methodology should be used. The goal of private sector researchers is to develop a method that is fast, inexpensive, reliable, and valid. If all these criteria are met, the researcher has performed a valuable task.

However, selecting a topic is a concern for many beginning researchers, especially those writing term papers, theses, and dissertations. The problem knows where to start. Fortunately, there are virtually unlimited sources available in searching for a research topic; academic journals, periodicals, and newsweeklies, and everyday encounters can provide a wealth of ideas.  Although academic journals tend to publish research that is 12 to 24 months old (due to review procedures and backlog of articles),

The articles may provide ideas for research topics. Most authors conclude their research by discussing problems encountered during the study and suggesting topics that need further investigation. In addition, some journal editors build issues around individual research themes, which often can help in formulating research plans.

In addition to academic journals, professional trade publications offer a wealth of information relevant to mass media research. Research abstracts, located in most college and university libraries, are also valuable sources for research topics. These volumes contain summaries of research articles published in nearly every academic journal. 

2.2 Sources of Research Topics

2.2.1 Magazines and Periodicals

Many educators feel that publications other than professional journals contain only "watered-down" articles written for the general public. To some extent this is true, but these articles tend to eliminate the tedious technical jargon and are often good sources for problems and hypotheses. In addition, more and more articles written by highly trained professionals are appearing in weekly and monthly publications. These sources often provide interesting perspectives on complex problems and many times raise interesting questions that researchers can pursue.

2.2.2 Research Summaries

Professional research organizations irregularly publish summaries that provide a close look at the major areas of research in various fields. These summaries are often useful for obtaining information about research topics, since they survey a wide variety of studies. 

2.2.3 Everyday Situations

Each day we are confronted with various types of communication via broadcasting and print, interpersonal communication, public relations campaigns, and so forth. These confrontations can be excellent sources of research topics for the researchers who take an active role in analyzing them. What types of messages are produced? Why are they produced in a specific way? What effects are expected from the various types of communication? These and other questions may help develop a research idea. Significant studies based on questions arising from everyday encounters with the media and other forms of mass communication have covered investigations of television violence, layout of newspaper advertisements, advisory warnings on television programs, and approaches to public relations campaigns.

2.2.4 Archive Data

Data archives, such as the Inter-University Consortium for Political Research (ICPR) at the University of Michigan, the Simmons Target Group Index (TGI), the Galiup and Roper organizations, and the collections of Arbitron, Nielsen, and Birch media ratings data (Chapter 14), are valuable sources of ideas for researchers. The historical data are used by researchers to investigate questions different from those which the data were originally intended to address. For example, ratings books provide information about audience size and composition for a particular period in time, but other researchers may use the data for historical tracking, prediction of audiences in the future, the changes in popularity of types of stations and/or programs, and the relationship between audience ratings and advertising revenue generated by individual stations or an entire market. This process, known as secondary analysis, has become a major research approach because of the time and resource savings it affords.

Secondary analysis provides an opportunity for researchers to evaluate otherwise unavailable data.  Secondary analysis may be defined as: [the] reuse of social science data after they have been put aside by the researcher who gathered them. The reuse of the data can be by the original researcher or someone uninvolved in any way in the initial research project. The research questions examined in the secondary analysis can be related to the original research endeavor or quite distinct from it.

2.2.5 Advantages of Secondary Analysis

Ideally every researcher should conduct a research project of some magnitude to learn about design, data collection, and analysis. Unfortunately, this ideal situation does not exist. Modern research is simply too expensive. In addition, because survey methodology has become so complex, it is rare to find one researcher, or even a small group of researchers, who are experts in all phases of large studies.

Secondary analysis is one research alternative that solves some of these problems. There is almost no expense involved in using available data. There are no questionnaires or measurement instruments to construct and validate salaries for interviewers and other personnel are nonexistent, and there are no costs for subjects and special equipment. The only expenses entailed in secondary analysis are those for duplicating materials — some organizations provide their data free of charge — and computer time. Data archives are valuable sources for empirical data. In many cases, archive data provide researchers with information that can be used to help answer significant media problems and questions,

Secondary analysis has a bad connotation for some researchers, especially those who are unfamiliar with its potential. Although researchers can derive some benefits from developing questionnaires and conducting a research project using a small and often unrepresentative sample of subjects, this type of analysis rarely produces results that are externally valid. The argument here is that in lieu of conducting a small study that has limited (if any) value to other situations, researchers would benefit from using data that have been previously collected.

Another advantage of secondary analysis is that data allow researchers more time to further understand what has been collected. All too often research is conducted and after a cursory analysis of the data for publication or report to management, the data are set aside, never to be touched again. It is difficult to completely analyze all data from any research study in just one or two studies, yet this procedure is followed in both the academic and private sectors.

Arguments for secondary analysis come from a variety of researchers It is clear that the research method provides excellent opportunities to produce valuable knowledge. The procedure, however, is not free from criticism.

2.2.6 Disadvantages of Secondary Analysis

Researchers who use secondary analysis are limited to the types of hypotheses or research questions that can be investigated. The data already exist, and since there is no way to go back for further information, researchers must keep their analyses within the boundaries of the type of data originally collected.

Researchers conducting secondary analysis studies also may face the problems of using data that were poorly collected, inaccurate, or flawed. Many studies do not include information about the research design, sampling procedures, weighting of subjects' responses, or other peculiarities. Perhaps it is suspected that some of the data were fabricated. Large research firms tend to explain their procedures in detail.

Although individual researchers in mass media have begun to make their data more readily available, not all follow adequate scientific procedures. This may seriously affect a secondary analysis.

Before selecting a secondary analysis approach, researchers need to consider the advantages and disadvantages. However, with the increased use of secondary analysis, some of the problems associated with research explanations and data storage are being solved.

2.3 Determining Topic Relevance

Once a basic research idea has been chosen or assigned, the next step is to ensure that the topic has merit. This step can be accomplished by answering eight basic questions.

Question 1: Is the Topic Too Broad?

Most research studies concentrate on one small area of a field; few researchers attempt to analyze an entire field in one study. There is a tendency, however, for researchers to choose topics that, while valuable, are too broad to cover in one study — for example, "the effects of television violence on children," or "the effects of mass media information on voters in a president's trial election."

To avoid this problem, researchers usually write down their proposed title as a visual starting point and attempt to dissect the topic into small questions.

Question 2: Can the Problem Really Be Investigated?

Aside from considerations of broadness, a topic might prove unsuitable for investigation simply because the question being asked has no answer, or at least cannot be answered with the facilities and information available. For example, a researcher who wants to know how people who have no television receiver react to everyday interpersonal communication situations must consider the problems of finding subjects without at least one television set in the home. Some may exist in remote parts of the country, but the question is basically unanswerable due to the current saturation of television. Thus the researcher must attempt to reanalyze the original idea in conformity with practical considerations.

Another point to consider is whether all terms of the proposed study are definable. Remember that all measurable variables must be operationally defined. A researcher who is interested in examining youngsters' use of the media needs to come up with a working definition of the word youngsters to avoid confusion. Potential problems can be eliminated if an operational definition is stated: "Youngsters are children between the ages of 3 and 7 years."

One final consideration is to review available literature to determine whether the topic has been investigated. Were there any problems in previous studies? What methods were used to answer the research questions? What conclusions were drawn?

Question 3: Are the Data Susceptible to Analysis?

A topic does not lend itself to productive research if it requires collecting data that cannot be measured reliably and validly. In other words, a researcher who wants to measure the effects of not watching television should consider whether the information about the subjects' behavior will be adequate and reliable, whether the subjects will answer truthfully, what value the data will have once gathered, and so forth. Researchers also need to have enough data to make the study worthwhile. It would be inadequate to analyze only 10 subjects in the "television turn-off" example, since the results could not be generalized with regard to the entire population.

Another consideration is the researcher's previous experience with the statistical method selected to analyze the data. That is, does he or she really understand the proposed statistical analysis? Researchers need to know how the statistics work and how to interpret the results. All too often researchers design studies involving advanced statistical procedures that they have never used. This tactic invariably creates errors in computation and interpretation. Research methods and statistics should not be selected because they happen to be popular or because a research director suggests a given method, but rather because they are appropriate for a given study and are understood by the person conducting the analysis. A common error made by beginning researchers is to select a statistical method without understanding what the statistic actually produces. Using a statistical method without understanding what the method produces is called the law of the instrument. It is much wiser to do simple frequencies and percentages and understand the results than to try to use a high-level statistic and end up totally confused.

Question 4: Is the Problem Significant?

Before a study is conducted, the researcher must determine whether it has merit, that is, whether the results will have practical or theoretical value. The first question to ask is: Will the results add knowledge to the information already available in the field? The goal of all research is to help further the understanding of the problems and questions in the field of study; if a study does not do this, it has little value beyond the experience the researcher acquires from conducting it. This does not mean that all research has to be earth-shattering. Many investigators, however, waste valuable time trying to develop monumental projects when in fact the smaller problems are of more concern.

A second question is what is the real purpose of the study? This is important because it helps focus ideas. Is the study intended for a class paper, a thesis, a journal article, a management decision? Each of these projects has different requirements concerning background information needed, amount of explanation required, and detail of results generated. For example, applied researchers need to determine whether any useful action based on the data will prove to be feasible, as well as whether the study will answer the question(s) posed by management.

Question 5: Can the Results of the Study Be Generalized?

For a research project to have practical value — to be significant beyond the immediate analysis — it must have external validity; that is, one must be able to generalize from it to other situations. For example, a study of the effects of a small-town public relations campaign might be appropriate if plans are made to analyze such effects in several small towns, or if it is a case study not intended for generalization; however, such an analysis has little external validity.

Question 6: What Costs and Time are Involved in the Analysis?

In many cases the cost of a research study is the sole determinant of the feasibility of a project. A researcher may have an excellent idea, but if costs would be prohibitive, the project must be abandoned. A cost analysis must be completed very early on. It does not make sense to develop specific designs and the data-gathering instrument for a project that will be canceled because of lack of funds. Sophisticated research is particularly expensive: costs may easily exceed 50,000 LE for one project.

A carefully itemized list of all materials, equipment, and other facilities required is necessary before beginning a research project. If the costs seem prohibitive, the researcher must determine whether the same goal can be achieved if costs are shaved in some areas. Another possibility to consider is financial aid from graduate schools, funding agencies, local governments, or other groups that subsidize research projects. In general, private sector researchers are not severely constrained by expenses; however, they must adhere to budget specifications provided by management.

Time is also an important consideration in research planning. Research studies must be designed in such a way that they can be completed in the amount of time available. Many studies have failed because not enough time was allotted for each research step, and in many cases, the pressure created by deadlines creates problems in producing reliable and valid results (for example, failure to provide alternatives if the correct sample of people cannot be located).

Question 7: Is the Planned Approach Appropriate to the Project?

The most marvelous research idea may be greatly, and often needlessly, hindered by a poorly planned method of approach. For example, a researcher who wished to measure any change in attendance at movie theaters that may have accompanied the increase in television viewing in one city could mail questionnaires to a large number of people to determine how media habits have changed during the past few years. However, the costs of printing and mailing questionnaires, plus follow-up letters and possibly phone calls to increase the response rate, might prove prohibitive.

Could this study be planned differently to eliminate some of the expense? Possibly, depending on the purpose of the study and the types of questions planned. The researcher could collect the data by telephone interviews to eliminate printing and postage costs. Some questions might need reworking to fit the telephone procedure, but the essential information could be collected. A close look at every study is required to plan the best approach.

Question 8: Is There Any Potential Harm to the Subjects?

Researchers must carefully analyze whether the project may cause any physical or psychological harm to the subjects under evaluation. For example: Will respondents be frightened in any way? Will they be required to answer embarrassing questions or perform embarrassing acts that may create adverse reactions? Is there any possibility that the exposure to the research conditions will have lasting effects? Prior to the start of most public research projects involving human subjects, detailed statements explaining the exact procedures involved in the research are required to ensure that subjects will not be injured in any way. These statements are intended to protect unsuspecting subjects from being exposed to harmful research methods.

Underlying all eight steps in the research topic selection process is validity (Chapter 3). In other words, are all of the steps (initial idea to data analysis and interpretation) the correct ones to follow in trying to answer the question(s)?

2.4 Reviewing the Literature

Researchers who conduct studies under the guidelines of scientific research never begin a research project without first consulting available literature. The review provides information about what was done, how it was done, and what results were generated. Experienced researchers consider the literature review as one of the most important steps in the research process because it not only allows them to learn from (and eventually add to) previous research data but also saves time, effort, and money. Failing to conduct a literature review is as detrimental to a project as failing to address any of the other steps in the research process.

Before any project is attempted, researchers ask the following questions:

1.      What type of research has been done in the area?

2.      What has been found in previous studies?

3.      What suggestions do other researchers make for further study?

4.      What has not been investigated?

5.      How can the proposed study add to our knowledge of the area?

6.      What research methods were used in previous studies?

Answers to these questions will usually help define a specific hypothesis or research question.

2.5 Stating a Hypothesis or Research Question

After a general research area has been identified and the existing literature reviewed, the researcher must state the problem as a workable hypothesis or research question. A hypothesis is a formal statement regarding the relationship between variables, and it is tested directly. The predicted relationship between the variables is either true or false. On the other hand, a research question is a formally stated question intended to provide indications about something, and it is not limited to investigating relationships between variables. Research questions are generally used in situations where a researcher is unsure about the nature of the problem under investigation. The intent is merely to gather preliminary data. However, testable hypotheses are often developed from information gathered during the research question phase of a study.

2.6 Research and Experimental Design

Different research approaches are required. Some questions call for a survey methodology via telephone or mail; others are best answered through in-person interviews. Still other problems necessitate a controlled laboratory situation to eliminate extraneous variables. The approach selected by the researcher depends on the goals and purpose of the study and how much money is available to conduct the analysis. Even projects that sound very simple may require a highly sophisticated and complex research approach.

The terms research design and experimental design have become interchangeable to refer to the process involved in developing or planning a research project. Some researchers prefer to use research design to describe nonlaboratory projects, and experimental design only for projects conducted in a laboratory setting. In this book, the terms are used interchangeably because countless arguments can be raised about whether or not a research project is an "experiment," and the relationship between "laboratory" and "experiment." That is, must an "experiment" be conducted in a controlled laboratory situation to be called an "experiment"?

Research and experimental design are essentially blueprints, or sets of plans, for collecting information. The ideal design collects a maximum amount of information with a minimal expenditure of time and resources. Depending on the circumstances, a design may be brief or very complicated; there are no specific guidelines concerning the amount of detail required for a design. However, all designs incorporate the steps in the process of collecting and analyzing the data.

Researchers must determine how the data will be collected and analyzed before beginning a research project. Attempting to force a study to follow a particular approach or statistic after the data have been gathered only invites error. For example, a director of marketing for a large shopping mall was interested in finding out more about the customers who shopped at the mall (for example, where they lived and how often they shopped at the mall). With very little planning, she designed a simple questionnaire to collect the information. However, the respondents' possible answers, or response choices, to each of the questions were inadequate and the questionnaire inappropriately designed for any type of summary analysis. Thus, the director of marketing was stuck with thousands of useless questionnaires.

All research — from very simple surveys of only a few people to nationwide studies covering complex issues — requires a design of some type. All procedures, including variables, samples, and measurement instruments, must be selected or designed in light of their appropriate-ness to the hypotheses or research questions, and all items must be planned in advance.

There are four characteristics of research design that should be noted if a study is to produce reliable and valid results:

1. Naturalistic setting: For the results of any project to have external validity, the study must be conducted under normally

encountered environmental conditions. This means that subjects should be unaware of the research situation, if possible; that phenomena should not be analyzed in a single session; and that normal intervening variables, such as noise, should be included in the study. Also, long-term projects are more conducive to a naturalistic atmosphere than short-term studies.

2. Clear cause-and-effect relationships: The researcher must make every effort to control intervening or spurious independent/dependent variable relationships (Chapter 3). The results of a study can be interpreted with confidence if and only if all confounding effects are identified.

3. Unobtrusive and valid measurements: There should be no perceptible connection between the communication presented to subjects and the measurement instruments used. Subjects tend to answer questions differently if they can identify the purpose of the study. Also, the study should be designed to assess both immediate and long-term effects on the subjects.

To assure the validity of the measurements used, a sample should be large enough to allow detection of minor effects or changes (Chapter 4). Additionally, the selection of dependent variables should be based on their relevance to the study and the researcher's knowledge of the area, not on convenience.

4. Realism: A research design must above all be realistic. This necessitates a careful consideration of the availability of time, money, personnel to conduct the study, and researchers who are competent in the proposed research methodology and statistical analysis.

Once the research design has been properly developed, researchers should pretest as many phases of the project as possible. A pretest of the questionnaire, and a check for errors in the measurement instruments) and equipment will help determine if significant problems are present. A trial run or pilot study (a small-scale version of the planned research project) is recommended, but is not always necessary or possible. The mall marketing director in the previous example could have saved a great deal of time and money by running a pilot study using 10 or 20 mall shoppers. She would have quickly discovered that the questionnaire did not produce the desired results.